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Overview 


Low-income preschool children face risks to their social-emotional development that can affect 
them later on. Although there are promising approaches to promoting preschoolers’ social-emotional 
skills, the evidence base is limited, particularly on their effectiveness when implemented at scale. 

The Head Start CARES demonstration evaluated the effects of three distinct classroom-based 
approaches to enhancing children’s social-emotional development on a large scale. The Incredible 
Years Teacher Training Program focuses on teachers’ management of the classroom and of 
children’s behavior. Preschool PATHS uses structured lessons to help children learn about emo- 
tions and interact with peers appropriately. Tools of the Mind-Play is a one-year version of the 
Tools curriculum that promotes children’s learning through structured “make-believe” play. A 
comprehensive professional development package (including teacher training, ongoing coaching, 
and related technical assistance) supported delivery of the enhancements over the course of one year. 

The demonstration was conducted with 17 Head Start grantees that generally represent the diversity 
of Head Start settings nationally. Head Start CARES rigorously evaluated the impacts of the interven- 
tions, or “enhancements,” by randomly assigning approximately 100 Head Start centers within the 
grantees to a program group that received one of the interventions or to a control condition without 
any of them. The estimated impacts should be interpreted as the effects of the enhancements beyond 
any effects of the existing Head Start program in these classrooms. Head Start CARES tested each 
enhancement’s impacts on teachers’ practices and on children’s outcomes in the spring of the 
preschool year, comparing those impacts with the team’s theory of change for each approach. 

Key Findings 

• All three enhancements changed observed teachers’ practices in the expected ways, confirming 
that their theoretical differences were reflected in classroom implementation. Impacts on class- 
room climate were fewer and less consistent. 

• Two of the three enhancements showed consistent positive impacts on a range of children’s 
social-emotional outcomes in preschool, although not as predicted. 

o As expected, PATHS showed small to moderate improvements in children’s knowledge 
and understanding of emotion (emotion knowledge), social problem-solving skills, and so- 
cial behaviors. 

o The Incredible Years improved children’s emotion knowledge, social problem-solving 
skills, and social behaviors. It did not produce expected impacts on children’s problem be- 
havior and executive function (except for highest-risk children). 

o Tools of the Mind - Play did not demonstrate expected impacts on executive function or 
self-regulation; it produced only positive impacts on emotion knowledge. 

• There was no consistent evidence that these enhancements improved children’s pre-academic 
skill s during preschool, although those skills were not key targets of the enhancements. While 
based on very limited information, there were virtually no impacts on outcomes in kindergarten 
as reported by teachers and parents. 

• Findings show that evidence-based approaches can improve preschoolers’ social-emotional 
competence when implemented at scale with appropriate supports. 


v 
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Executive Summary 


Low-income preschool children experience greater risks to their social and emotional develop- 
ment than their more affluent peers. 1 These gaps are observed before children begin their formal 
schooling, and they remain or increase during the elementary school years. 2 Since social and 
emotional risks may compromise children’s future mental health and peer relationships, 
addressing them is important in its own right. 3 In addition, social-emotional competence may 
aid learning: children who have difficulty regulating their emotions and behaviors have been 
found to receive less instruction, to be less engaged in and less positive about their role as 
learners, and to have fewer opportunities for learning from peers. 4 

The Head Start CARES (Classroom-based Approaches and Resources for Emotion and 
Social skill promotion) demonstration tests three distinct approaches to enhancing children’s 
social-emotional development on a large scale within the Head Start system — the largest 
federally funded early-childhood education program in the United States. Conceived and 
sponsored by the Office of Head Start and the Office of Planning, Research and Evaluation in 
the Administration for Children and Families within the U.S. Department of Health and Human 
Services, the Head Start CARES demonstration was conducted by MDRC, a nonprofit, nonpar- 
tisan education and social policy research organization, in collaboration with MEF Associates 
and several academic partners. 

The three social-emotional approaches tested in Head Start CARES were called “en- 
hancements” because they complemented and enriched classroom practices that already existed. 
The effects, or “impacts,” of the enhancements were rigorously evaluated by randomly assign- 
ing approximately 100 Head Start centers to one of the three enhancements (the program group) 
or to a control group that continued with “business as usual.” Therefore, estimated impacts 
should be interpreted as the effects of the enhancements over and above any effects of the 
existing Head Start program in these sites. 


'Dodge, Pettit, and Bates (1994); Brooks-Gunn, Duncan, and Aber (1997). The Center on the Social and 
Emotional Foundations for Early Learning defines social-emotional development as the “developing capacity 
of the child from birth through five years of age to form close and secure adult and peer relationships; experi- 
ence, regulate, and express emotions in socially and culturally appropriate ways; and explore the environment 
and leam — all in the context of family, community, and culture” (Center on the Social and Emotional 
Foundations for Early Learning, 2008; Yates et al., 2008). 

Alexander, Entwisle, and Kabbani (2001); Brooks-Gunn, Duncan, and Aber (1997); Dodge, Pettit, and 
Bates (1994); Entwisle and Hayduk (1988); Hamre and Pianta (2001); Ladd, Buhs, and Troop (2002). 

3 Briggs-Gowan and Carter (2008); Caspi, Moffitt, Newman, and Silva (1996); Schwartz et al. (1999). 

4 Ladd, Birch, and Buhs (1999); McClelland, Morrison, and Holmes (2000); Raver, Garner, and Smith- 
Donald (2007). 
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As described in an earlier report on the Head Start CARES demonstration, 5 a compre- 
hensive professional development system for teachers — including four to six training sessions, 
weekly coaching sessions in the classroom, a “real-time” management infonnation system 
(MIS) to support monitoring, and technical assistance — supported the scale-up of the en- 
hancements around the country. The teacher training and coaching were generally implemented 
as intended, supporting satisfactory implementation (a rating of 3 on a scale of 1 to 5) of the 
social-emotional enhancements in Head Start classrooms and leading to the expected influences 
on teachers’ practices, which are described below. Thus, it appears that the demonstration 
ensured a fair test of large-scale implementation of the three enhancements, providing a sound 
basis for evaluating their impact on children and classrooms in the Head Start system. 

This report presents the impacts of the three enhancements tested in the Head Start 
CARES demonstration. It focuses on outcomes in the spring of the preschool year for (1) 
teachers’ practices; (2) the climate of the classroom; (3) children’s behavior regulation, execu- 
tive function skills, 6 knowledge and understanding of emotions (“emotion knowledge”), and 
social problem-solving skills; and (4) children’s learning behaviors and social behaviors. In 
addition to changing teachers’ practices, two of the three enhancements had consistent positive 
impacts on a range of children’s social-emotional outcomes, although not necessarily in ways 
that would be expected according to the theories of change that the CARES team developed. 7 

The Head Start CARES study thus demonstrates that preschool children’s social- 
emotional outcomes can be improved when evidence-based approaches — that is, approaches 
that have been shown to result in differences in children’s social and emotional outcomes — are 
implemented at scale with appropriate supports. The report also includes an exploratory set of 
findings, which have not been previously tested for these enhancements, about whether the 
enhancements might improve children’s early academic skills in preschool and whether they 
have any sustained effects as preschool children make the transition to elementary school. 


The Three Program Enhancements Tested in Head Start CARES 

The three social-emotional enhancements that Head Start CARES tested were “The Incredible 
Years Teacher Training Program,” “Preschool PATHS” (Promoting Alternative Thinking 
Strategies), and “Tools of the Mind— Play.” These enhancements were selected for testing 


5 Mattera, Lloyd, Fishman, and Bangser (2013). 

Executive function, also known as cognitive regulation, in early childhood includes working memory (or 
the ability to keep a number of pieces of information in the mind at once), set-shifting (or the ability to flexibly 
shift between pieces of information), and inhibition (or the ability to stop or repress an immediate response). 

7 The CARES team developed the theory of change for each enhancement based on the training and cur- 
ricular materials and research papers from each one. 
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because each one was thought to exemplify a distinct theory of change for improving children’s 
social-emotional development, and because prior empirical evidence had shown that they were 
efficacious with low-income children in smaller-scale tests. In addition, these enhancements had 
packaged materials and training guidelines available to improve their potential for replication 
with fidelity to the original model in Head Start centers across the country, and they could be 
implemented in Head Start classrooms for all children in those classrooms (rather than for just a 
high-risk subsample) over a single year of implementation. Head Start CARES was the first 
large-scale test of these enhancements. 

The Incredible Years Teacher Training Program focuses on training teachers to cre- 
ate an organized classroom climate that supports children’s ability to regulate their own behav- 
ior in the context of positive teacher-child relationships . 8 The enhancement includes strengthen- 
ing and promoting positive teacher-child relationships, classroom organization (rules and 
predictable routines), clear and consistent limit-setting, praise and incentives to motivate 
students’ learning, and proactive discipline strategies. For instance, during “circle time” in an 
Incredible Years classroom, the teacher might say, “I really like the way Juan is sitting with his 
hands in his lap,” instead of mentioning the children who are still playing and not paying 
attention to the teacher . 9 

Preschool PATHS focuses on training teachers to use clearly outlined lessons and 
teaching strategies to improve children’s emotion knowledge and social problem-solving skills, 
including the ability to recognize, understand, and communicate about emotions; interpret 
difficult social situations; and select from a set of various competent solutions to such situations. 
PATHS provides teachers with weekly lessons that are taught during circle time, and teachers 
can also use other activities during the rest of the school day to give children opportunities to 
practice the skills they have learned. In a PATHS classroom, teachers talk about their feelings 
and encourage children to think about their and others’ feelings. For example, if two children 
are playing together in the block area, the teacher might say, “How do you think An n felt when 
Neveah gave her a hug? That’s right, she felt happy! How do you look when you feel happy? 
What makes you feel happy?” 


8 The Teacher Training Program is one of three Incredible Years programs; the other two are the child- 
focused Dinosaur School and the parent-focused Parent Program. 

9 During circle time, which often occurs at the beginning of the day in preschool classrooms, the children 
and the teacher gather together in a circle for a particular set of activities such as greeting each other, singing a 
song, or counting the number of children who are in school that day. 
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Tools of the Mind— Play is a one-year adaptation of Tools of the Mind. 10 It trains 
teachers to support children’s planning and enacting “make-believe” (or “pretend”) play and 
role-playing games to strengthen children’s ability to regulate their emotions and behavior. 11 A 
central component of Tools is a daily 50-minute period devoted to adult-supported pretend play. 
This component is organized and scaffolded by teachers to enhance children’s ability to plan for 
and understand various social roles — such as the role of family members — while enhancing 
their memory, ability to focus their attention, and understanding of their own and their peers’ 
perspectives. 12 For example, in a Tools classroom, a child might draw a picture showing that 
she intends to play house and will be the mother. The teacher would then help the child write 
out and expand on her plans. Then, while the child is playing, the teacher might come over to 
help her expand the role-play even further, asking questions such as, “What might you need 
before you are able to cook dinner?” and “How would you get to the grocery store?” 

Figure ES. 1 shows the pathway through which the CARES enhancements are expected 
to have an influence on teachers’ practices, classroom climate, children’s social-emotional 
skills, and children’s behavioral outcomes in the preschool year. Each enhancement has as its 
primary goal — and, therefore, expected key outcome — the improvement of some aspect(s) of 
children’s social-emotional competence. Each one attempts to achieve this goal by directly 
targeting teachers’ practices and, in the case of Preschool PATHS and Tools of the Mind, by 
delivering instructional content to the children as well. In each case, improved teachers’ 
practices (and instructional content) are thought to improve the quality of the classroom experi- 
ences, or classroom climate, for children. It is through changes in teaching practice (and perhaps 
content) and climate that the enhancements are thought to improve children’s social, emotional, 
and behavioral outcomes. 

Notably, however, while the enhancements share a core goal (improving children’s 
social-emotional competence), the CARES team hypothesized that each one has quite different 
mediating or intervening pathways to social-emotional competence for children, and that each 
one affects somewhat different aspects of children’s social-emotional competence. These 
hypotheses are discussed later with the findings for each enhancement, since each enhancement 
is being evaluated primarily in relation to the theory of change developed by the CARES team 
and the associated expected pattern of impacts. 


10 In Tools of the Mind-Play, teachers were trained for only one year in the model (instead of two years, as 
is typical in the Tools of the Mind program) and it was implemented as an enhancement to the existing 
curricula in the program sites. 

11 “Make-believe” is a form of high-level play in which children use their imaginations to role-play, pre- 
tend they are different characters, play out different stories, and enact various scenarios that rely on and 
encourage creativity. 

12 Scaffolding is the act of helping a child accomplish a challenging task or acquire a skill that is just be- 
yond the child’s current ability level. 


ESA 
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Head Start CARES Demonstration 


Intervention 


Incredible 
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Teacher 

Training 


Preschool 

PATHS 
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Figure ES.l 

Head Start CARES Theory of Change 


Improvements in Classroom 
Interactions 






The Design of the Head Start CARES Study and 
the Implementation Context 

Seventeen Head Start grantees were selected to reflect the geographic, racial, and ethnic 
diversity of the national Head Start population. Grantees were distributed fairly equally across 
four regions of the country, with four grantees in the Northeast, four in the West, three in the 
South, and six in the Midwest/Plains states. Grantees were distributed across community action 
agencies, stand-alone nonprofit entities, and large local school systems. 

For each grantee, groups of four or eight similar Head Start centers (in tenns of their ra- 
cial and ethnic mix, part-day or full-day programming, and four-year-old versus mixed-age 
classrooms) were randomly assigned to one of the three enhancements or to a control group that 
conducted “business as usual.” The resulting sample included 104 centers with 307 classrooms, 
with data collected on approximately nine children per classroom to assess impact (although all 
children in the classroom were treated similarly in tenns of their intervention group status). 
Children were dispersed widely to 648 schools when they entered kindergarten the next year. 

Data collection for the findings presented in this report included independent observa- 
tions of teaching practices and classroom climate, surveys of teachers on children’s learning and 
social behaviors, and independent direct assessments of children’s cognitive and social- 
emotional skills (executive function, knowledge of emotions, and social problem-solving skills), 
all in the spring of the Head Start year. 13 Since teachers and parents were aware when children 
participated in an enhancement and were not trained to assess children’s behaviors consistently, 
infonnation collected by trained observers and assessors was given priority in interpreting 
findings. Data on children’s outcomes in kindergarten were not collected using direct assess- 
ments, but a smaller set of outcomes was collected through teacher and parent reports. 

Findings 

Table ES.l highlights, with an “X,” statistically significant impacts (that is, impacts that 
are not likely to be a result of chance) in the areas of teacher practice, classroom climate, 
children’s social-emotional skills, and children’s learning and social behaviors in the spring of 
the preschool year. Dark areas of the table show primary, targeted outcomes for each enhance- 
ment; lighter gray areas show secondary outcomes. 


13 Baseline infonnation was also collected: classroom observations were collected in the spring before 
summer training in the varying social and emotional enhancement approaches; direct child assessments and 
teacher surveys on children were collected in the fall of the preschool year, once children were enrolled in 
centers. 
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Head Start CARES Demonstration 
Table ES.l 

Primary and Secondary Targeted Outcomes in Preschool, 
by Enhancement 



The Incredible 
Years 

Preschool PATHS 

Tools of the 
Mind-Play 

Outcome 

Trains teachers to 
create an organized 
classroom climate that 
supports children’s 
behavior regulation in 
the context of positive 
teacher-child 
relationships 

Trains teachers to use 
clearly outlined 
lessons to improve 
children’s ability to 
recognize and regulate 
emotions, define 
problems, and 
consider the 
consequences of 
various choices 

Trains teachers to use 
adult-supported make- 
believe play and other 
activities to strengthen 
children’s self- 
regulation 

Teacher practice (observational assessment) 

Classroom management 

x 



Social-emotional instruction 

X 

x 


Scaffolding 



X 

Classroom climate (observational assessment) 

Classroom organization 




Emotional support 




Instructional support 


X 


Literacy 



X 

Executive function and behavior regulation 8 

Executive function 




Behavior problems 




Emotion knowledge and social problem-solving skills (direct assessments) 

Emotion knowledge 

X 

X 

X 

Social problem-solving skills 

X 

X 


Learning and social behaviors (teacher reports) 

Learning behaviors 

X 

X 


Social behaviors 

X 

X 



NOTES: In each cell, “X” indicates that there was a statistically significant impact on that outcome. 
The dark gray cells represent primary targeted outcomes for the enhancement; the light gray cells 
represent secondary targeted outcomes. 

“Executive function was measured using direct assessments. Behavior problems were measured 
using teacher reports. 
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In sum, as discussed below and as shown in the table, the success of all three enhance- 
ments in changing teachers’ practices in the expected ways confirms that the enhancements 
were differentiated not only in theory but also as delivered in the classroom. Impacts on 
classroom climate were fewer and less consistent with the theory of change. 

Two of the three enhancements showed consistent improvements on a number of chil- 
dren’s social-emotional outcomes by the spring of the preschool year, although not necessarily 
in the expected ways. The positive impacts tended to be concentrated on children’s knowledge 
of emotions and their social skills, rather than on executive function and behavior regulation. 
Preschool PATHS produced the impacts most consistent with the CARES team’s theory of 
change; however, positive findings for The Incredible Years also suggest that there is more than 
one way for Head Start teachers to change their practices to improve children’s emotion 
knowledge, social problem-solving skills, and social behaviors in the preschool year. 


Impacts of “The Incredible Years” in the Preschool Year 

The Incredible Years’ explicit focus on teachers’ positive classroom management and behavior- 
al support strategies was expected to produce a more positive classroom climate, especially with 
regard to the level of emotional support and classroom organization. Children in these class- 
rooms were expected to demonstrate greater social-emotional competence, with the strongest 
effects on their behavior problems, executive function, and related engagement in classroom 
activities. Secondary benefits of The Incredible Years were expected for children’s emotional 
and social skills (that is, their understanding of the emotions and social problem-solving that are 
thought to underlie social interactions), as well as for teachers’ ratings of their students’ social 
skills. While the findings show that The Incredible Years did not reduce children’s behavior 
problems or improve their executive function skills, the enhancement did improve children’s 
emotional and social skills and their learning behaviors. 

• Incredible Years teachers were rated higher than their control group 
counterparts on some aspects of classroom management practices and 
social-emotional instruction. 

As expected, teachers in Incredible Years classrooms used more positive behavior man- 
agement practices (for example, rewarding good behavior and providing clear consequences), 
used fewer negative behavior-management practices (for example, yelling when there is 
misbehavior), and were better able to engage children’s attention. These were the central aspects 
of Incredible Years training. To a slightly lesser extent, this enhancement also improved 
teachers’ social-emotional instruction, even though the training materials did not focus on it 
explicitly or as extensively. 
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• Contrary to expectations, the positive impacts on teachers’ practices did 
not translate into wide-ranging impacts on the quality of the climate in 
Incredible Years classrooms. 

Although the Incredible Years enhancement improved teachers’ behavior management 
and reduced the negative climate in the classrooms, it did not improve the degree of warmth and 
sensitivity in the classroom, the extent to which classrooms were organized, or the level of 
instructional support that was provided. 

• For the Incredible Years research sample as a whole, no statistically 
significant impacts on children’s problem behaviors or on executive 
function skills were observed; however, the enhancement reduced prob- 
lem behaviors among the highest-risk children. 

The absence of impacts on problem behaviors and executive function skills for the In- 
credible Years research sample as a whole is notable (shown in Figure ES.2), since previous 
research in smaller-scale tests suggested that this enhancement may be especially effective for 
these aspects of children’s social-emotional competence. 14 However, teacher ratings did show 
that The Incredible Years reduced problem behaviors associated with acting out and hyperactiv- 
ity among children with high levels of behavior problems in the fall of the Head Start year. 
While this is only a single finding for a subgroup of the sample, it is consistent with research on 
The Incredible Years with children referred for mental health services, 15 who were a key focus 
in the development of the program. In addition, The Incredible Years improved learning 
behaviors, indicating that children in Incredible Years classrooms showed better engagement in 
learning tasks than their counterparts in control classrooms. 

• The Incredible Years produced small but statistically significant im- 
provements in children’s knowledge of emotions, social problem-solving 
skills, and social behaviors. 

According to the assessments conducted by trained interviewers, children in Incredible 
Years classrooms were beher at identifying emotional expressions and generating more compe- 
tent (including less aggressive) responses to stories about provocations, some of which were 
ambiguous, from other children — for example, a story in which a child knocks down another 
child’s tower of blocks or hits a child who is playing happily in the sandbox. In addition, 
teachers reported higher levels of social skills among children in Incredible Years classrooms. 


14 Raver et al. (2011); Morris et al. (2013). 

15 Herman, Borden, Reinke, and Webster-Stratton (2011); Reid, Webster-Stratton, and Hammond (2003); 
Webster-Stratton, Reid, and Hammond (2004); Webster-Stratton and Reid (2003). 
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Head Start CARES Demonstration 


Figure ES.2 

Impacts of The Incredible Years on Children’s Social and Emotional Outcomes 

in the Preschool Year 
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NOTES: Statistical significance levels are indicated as follows: *** = 1 percent; ** = 5 percent; 

* = 10 percent. 

This figure shows one measure as an exemplar for each outcome area. Executive function was 
measured by Pencil Tap; behavior problems were measured using the Behavior Problems Index; 
learning behaviors were measured using the Cooper-Farran Behavioral Rating Scale; emotion 
knowledge was measured by the Facial Emotions Task; social problem-solving skills were measured by 
the Challenging Situations Task (specifically, the Competent Response); and social behaviors were 
measured using the Social Skills Rating Scale. 

a The effect size is the change that is a result of the intervention, as specified in a standardized way 
that can be compared across different measures and outcomes within a study and across different 
studies. Technically, the effect size is computed as the impact divided by the standard deviation of the 
control group. 
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These social skills were not explicitly taught in these classrooms (as they were in Preschool 
PATHS classrooms, discussed below), suggesting that children can leam these critical skills in a 
number of ways. 

In sum, although many of the outcomes that The Incredible Years targets most directly 
were not affected (except for the highest-risk subgroup of children), there were impacts for 
children in both their knowledge of emotions and their ability to provide more competent and 
less aggressive responses to challenging social situations. Teachers’ reports of children’s social 
behaviors corroborated these findings. Since researchers theorize that the combination of 
emotion knowledge and social problem-solving skills can lead to lower levels of aggressive 
behavior toward peers, 16 it may bode well for the longer-term outcomes of children in Incredible 
Years classrooms. 

Impacts of “Preschool PATHS” in the Preschool Year 

Teachers in Preschool PATHS classrooms were expected to display higher levels of social- 
emotional instruction than teachers in the control group classrooms, including explicit lessons 
and activities that help children recognize and then respond to emotions appropriately. The 
delivery of lessons targeting children’s social-emotional skills was expected to lead to a more 
emotionally positive and well-organized classroom. Unlike The Incredible Years, which 
focused on children’s behavior regulation and problem behaviors, the focus of PATHS was on 
teaching children to understand their emotions and develop social problem-solving skills. This 
builds on the theory that these skills underlie children’s ability to respond positively, rather than 
aggressively, to their peers. The impact of PATHS on children’s behavior regulation and 
executive function skills was thought to be secondary to these outcomes. The findings, as 
presented in Figure ES.3, show that the impacts of PATHS were positive and quite consistent 
with the theory of change that the CARES team developed, especially with regard to some of its 
most proximal targets: teachers’ practices and children’s social-emotional skills. 

• Trained observers rated Preschool PATHS teachers higher than control 
group teachers on all the assessed aspects of social-emotional instruction 
that were the central focus of PATHS training. 

The observations of teachers in Preschool PATHS classrooms revealed that these 
teachers more often taught children about emotions, supported children’s expression and 
regulation of emotions, facilitated children’s understanding of their peers’ emotions and their 


16 Crick and Dodge (1994). 
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Head Start CARES Demonstration 


Figure ES.3 

Impacts of Preschool PATHS on Children’s Social and Emotional Outcomes 

in the Preschool Year 
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NOTES: Statistical significance levels are indicated as follows: *** = 1 percent; ** = 5 percent; 

* = 10 percent. 

This figure shows one measure as an exemplar for each outcome area. Executive function was 
measured by Pencil Tap; behavior problems were measured using the Behavior Problems Index; 
learning behaviors were measured using the Cooper-Farran Behavioral Rating Scale; emotion 
knowledge was measured by the Facial Emotions Task; social problem-solving skills were 
measured by the Challenging Situations Task (specifically, the Competent Response); and social 
behaviors were measured using the Social Skills Rating Scale. 

a The effect size is the change that is a result of the intervention, as specified in a standardized 
way that can be compared across different measures and outcomes within a study and across 
different studies. Technically, the effect size is computed as the impact divided by the standard 
deviation of the control group 
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social problem-solving skills, and supported children when they were distressed. These impacts 
were moderate to large in size, demonstrating that teachers were implementing the practices 
encouraged by PATHS to a great extent. 

• Preschool PATHS classrooms did not show the expected impacts on 
emotional support and classroom organization; they did score higher 
than control group classrooms on a measure of instructional support. 

The higher levels of social-emotional instruction by Preschool PATHS teachers were 
not accompanied by wanner interactions with children than was the case for teachers in control 
group classrooms. However, PATHS teachers did demonstrate higher levels of “concept 
development” (exemplified, for instance, by asking children “why” questions to support their 
higher-order thinking) and stronger quality of feedback (to support the quality of learning, not 
only the conect answer), both of which are thought to be integral to an effective learning 
environment for children. These results may reflect the fact that PATHS encouraged teachers to 
discuss emotions and social problem-solving when reading stories to children. 

• Preschool PATHS had small to moderate positive impacts on its prima- 
ry targets of children’s knowledge of emotions, social problem-solving 
skills, and social behaviors. 

Children in Preschool PATHS classrooms were better able than children in control 
classrooms to identify emotional expressions and to generate competent responses to peer 
provocation scenarios. Consistent with these findings, teachers reported higher levels of positive 
social behaviors (for example, cooperating with peers and effectively resolving conflicts with 
them) among children in PATHS classrooms, compared with children in control classrooms. 
The impacts on these outcomes were small to moderate in size, highly statistically significant, 
and consistent with findings in previous smaller-scale studies . 17 

• Preschool PATHS did not have consistent statistically significant im- 
pacts on children’s problem behaviors or executive function skills, which 
were not a central focus of the PATHS enhancement; PATHS did, how- 
ever, improve children’s learning behaviors as reported by teachers. 

Based on teachers’ reports of children’s behavior problems and interviewer assessments 
of children’s executive function skills, Preschool PATHS showed no statistically significant 
impacts on these outcomes. The only statistically significant impact on the secondary targets of 
PATHS was reflected in teachers’ reports of children’s learning behaviors. 


17 Bierman et al. (2008); Domitrovich, Cortes, and Greenberg (2007); Hamre, Pianta, Mashburn, and 
Downer (2012). 
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In sum, as expected from the theory of change that the CARES team developed, Pre- 
school PATHS trained teachers to deliver more effective instruction in social-emotional skills, 
and children in PATHS classrooms showed small to moderate improvements in their knowledge 
of emotions and their social problem-solving skills compared with children in control classrooms 
in the preschool year. Moreover, these effects on skills were complemented by preschool 
teachers’ reports of children’s improved social behaviors. Given that these emotional and social 
skills are thought to underlie children’s reduced aggression in social interactions, the findings 
indicate that PATHS met its goal of improving the building blocks of children’s social devel- 
opment in the preschool year. 


Impacts of “Tools of the Mind— Play” in the Preschool Year 

The Tools of the Mind enhancement that was implemented in Head Start CARES focuses on 
changing the way that “make-believe” (or playing “pretend”) and other learning experiences are 
structured and supported in the classroom. Teachers are expected to better scaffold children’s 
pretend play sequences and children’s interactions with peers during these play sequences. 
While there was no specific focus on teachers’ behavior management, Tools classrooms were 
expected to be better managed for two primary reasons: First, the restructuring of circle time 
and other activities in order to reduce whole-group instruction would, it was believed, make it 
easier to keep children engaged. Second, it was thought that the focus on self-regulation through 
play would lead to children’s better behavior and fewer classroom management issues. The 
primary expected impacts for Tools were in children’s executive function skills and learning 
behaviors. Children’s problem behavior as well as their understanding of emotions and their 
social problem-solving skills were presumed to be secondary. 

Notably, despite the reduction to a one-year enhancement with a focus only on the pri- 
mary elements of the program, Tools of the Mind was still a somewhat more complex en- 
hancement to implement than either The Incredible Years or Preschool PATHS. As discussed in 
the first report on Head Start CARES , 18 Tools classrooms had lower levels of implementation 
fidelity than did the other two enhancements. 

• Teachers in Tools of the Mind classrooms engaged in more scaffolding 
of children’s pretend play and peer interactions than teachers in the 
control group did. 

Consistent with its central focus, Tools of the Mind had moderate-sized impacts on 
teachers’ scaffolding of pretend play and peer interactions. No statistically significant impacts 


ls Mattera, Lloyd, Fishman, and Bangser (2013). 
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were observed on teachers’ classroom management practices or on their social-emotional 
instruction. 

• Tools of the Mind classrooms did not demonstrate higher levels of emo- 
tional support, classroom organization, or instructional support than 
control classrooms, although they did score higher than control class- 
rooms on literacy instruction. 

Tools’ impact on literacy instruction was not necessarily expected, but it is consistent 
with the fact that some of the Tools activities (such as asking children to write out their plans for 
pretend play) involve a focus on children’s literacy. 

• Children in Tools of the Mind classrooms did not demonstrate better 
executive function skills or better behavior regulation (fewer problem 
behaviors) than children in the control group. 

The play planning and pretend play sequences in Tools of the Mind intentionally target 
children’s behavior regulation and executive function skills. However, based on both teacher 
ratings of problem behaviors and trained interviewers’ direct assessment of tasks, no statistically 
significant impacts on any of these measures of children’s social-emotional competence were 
observed (as shown in Figure ES.4). 

• Children in Tools of the Mind classrooms demonstrated slightly greater 
knowledge of emotions, but not better social problem-solving skills, than 
children in control classrooms. 

Although the children did not receive explicit lessons to support their understanding 
of emotions, they would likely have had an opportunity to improve emotion knowledge 
through play and interpreting their peers’ emotional expressions in the Tools play sequences. 
However, no impacts were observed on other aspects of social skills, such as peer interactions 
and children’s ability to generate more competent social problem-solving solutions. 

In sum, as expected, Tools teachers were better able to scaffold children’s pretend 
play and peer interactions when compared with control teachers. However, children in 
Tools classrooms did not demonstrate the expected impacts on their executive function or 
self-regulation skills. While the children did show better knowledge of emotions, this was 
not accompanied by more competent social problem-solving skills. Earlier research has 
shown that it is the package of these skills (knowledge of emotions and the ability to 
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Head Start CARES Demonstration 


Figure ES.4 

Impacts of Tools of the Mind on Children’s Social and Emotional Outcomes 

in the Preschool Year 
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NOTES: Statistical significance levels are indicated as follows: *** = 1 percent; ** = 5 percent; 
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This figure shows one measure as an exemplar for each outcome area. Executive function was 
measured by Pencil Tap; behavior problems were measured using the Behavior Problems Index; 
learning behaviors were measured using the Cooper-Farran Behavioral Rating Scale; emotion 
knowledge was measured by the Facial Emotions Task; social problem-solving skills were measured 
by the Challenging Situations Task (specifically, the Competent Response); and social behaviors 
were measured using the Social Skills Rating Scale. 

a The effect size is the change that is a result of the intervention, as specified in a standardized 
way that can be compared across different measures and outcomes within a study and across 
different studies. Technically, the effect size is computed as the impact divided by the standard 
deviation of the control group 

b Executive function, for which the effect size is 0, was a primary targeted outcome. 
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generate competent responses to peer provocation) that is critical for better social interac- 
tions later in childhood. 19 


Impacts on Pre-Academic and Kindergarten Outcomes 

Impacts on Pre-Academic Outcomes in the Preschool Year 

• There was no consistent evidence that the three social-emotional en- 
hancements tested in Head Start CARES improved children’s pre- 
academic skills during preschool. 

While social-emotional development is important in its own right (given potential asso- 
ciations with long-term outcomes like aggression and delinquency), 20 this study also examined 
whether any gains in the social-emotional domain might extend to children’s pre-academic 
skills during the preschool year, even though such skills were not targeted directly and prior 
evidence on such effects is inconsistent. This analysis showed that none of the enhancements 
produced consistent impacts on direct assessments of children’s pre-academic skills during the 
Head Start year. 

Impacts on Teacher- and Parent-Reported Kindergarten Outcomes 

The Head Start CARES study also explored the extent to which any impacts on social- 
emotional outcomes during the Head Start year (1) were sustained into kindergarten; and 
(2) might lead to improved academic skills, as well as reductions in related schooling outcomes 
such as grade retention and special education. Prior research suggests that the benefits of The 
Incredible Years and Preschool PATHS on children’s knowledge of emotions and their social 
problem-solving skills might result in reduced aggression with peers and better peer interactions 
during middle childhood. However, since earlier evidence of sustained effects of these en- 
hancements into kindergarten is limited, and since information on outcomes in kindergarten was 
collected only from teachers and parents (who are not trained to rate behavior consistently), 
analyses of these data are considered preliminary. 


19 Dodge, Pettit, Bates, and Valente (1995); Dodge and Price (1994); Orobio de Castro, Bosch, Veerman, 
and Koops (2003). 

20 Briggs-Gowan and Carter (2008); Caspi, Moffitt, Newman, and Silva (1996); Dodge, Pettit, and Bates 
(1994). 
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• Despite the observed impacts on children’s social-emotional outcomes 
during the preschool year, none of the three enhancements appeared to 
have consistent impacts on children’s outcomes in kindergarten as 
reported by teachers and parents. 

No impacts were observed on parent-reported behavior problems or social behaviors in 
kindergarten, but parents of children who had been in Incredible Years classrooms reported 
higher receipt of special education services. There were virtually no impacts on kindergarten 
teachers’ reports of behavior problems, social behaviors, academic skills, or receipt of special 
services. However, one finding worth noting for children who were assigned to Preschool 
PATHS is that kindergarten teachers indicated statistically significantly lower levels of expected 
grade retention (from 7 percent of children coming from control group classrooms to less than 1 
percent of children coming from PATHS classrooms). This finding is potentially important but 
requires replication and further follow-up, given the general pattern of a lack of impacts across 
outcomes, the lack of a corroborating evidence base from other research, and the limited 
measurement of child outcomes in kindergarten in this study. 


Conclusion 

As the first large-scale test of three social-emotional enhancements in Head Start programs 
nationally, this study provides critical information to the field about how interventions seeking 
to support children’s social-emotional development can focus their training of teachers. The 
findings suggest, perhaps most important, that scaled-up, evidence-based models can produce 
impacts in the social-emotional domain during the preschool year of nearly the same magnitude 
as those from smaller-scale, more controlled studies when the models are supported by strong, 
comprehensive professional development. In addition, these findings suggest that more than one 
evidence-based, social-emotional approach (and, in particular, The Incredible Years or Pre- 
school PATHS) may be effective in meeting Head Start’s goal of improving children’s social- 
emotional development. Finally, these findings also suggest some key challenges for the field 
moving forward. 

First, the improvements in teachers’ practices and children’s skills emerged when well- 
designed, evidence-based models with prepared written materials were supported by high- 
quality and ongoing training and coaching of teachers and a real-time MIS. This comprehensive 
infrastructure allowed teachers to leam the enhancement content in training, return to their 
classrooms, practice using program strategies with continuous feedback from their coaches, and 
return to training to reflect on their implementation. In addition, coaches and trainers continu- 
ously reported on implementation through the MIS, allowing for technical assistance to be 
provided in order to address implementation challenges as they were occurring in real time. 
This constellation of supports led to improvements in teacher practices, as well as in some 
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social-emotional outcomes, that were nearly as large as those from the less dispersed, more 
controlled efforts that had been conducted previously. 

Second, improving children’s understanding of emotions and their social problem- 
solving skills and associated social behaviors may be accomplished either by supporting 
teachers’ positive classroom management practices (as was done in The Incredible Years) or 
their explicit teaching of emotional understanding and social skills through a more lessons- 
based approach (as was done in Preschool PATHS). Head Start programs may therefore have 
some options in selecting models that best meet the needs of their teachers and centers, with 
likely benefits for the children they are serving. 

That said, while it is clearly possible to improve children’s social skills and social be- 
haviors, it may be more challenging to improve their behavior regulation and related executive 
function skills. None of the models that was implemented in Head Start C ARES improved these 
outcomes for children. If, as has been suggested , 21 children’s academic achievement over the 
long tenn is related in part to their “grit” and persistence (which are likely to be influenced by 
children’s underlying behavior regulation and executive function skills), the field still has some 
way to go in identifying approaches to support children’s development in this area. 

Third, assessing teacher practices may be important for Head Start centers that are in- 
terested in improving children’s social-emotional development. The Incredible Years and 
Preschool PATHS improved teachers’ practices as well as children’s emotion knowledge, social 
problem-solving skills, and social behaviors, without corresponding impacts on CLASS 
scores . 22 As such, the findings suggest the importance of assessing and strengthening specific 
teacher practices in efforts to enhance children’s social-emotional development. 

Finally, the long-term academic (or social) benefits of investing in social-emotional 
development are not yet clear. The findings presented in this report show that children in Head 
Start centers that implemented PATHS and The Incredible Years had better social-emotional 
outcomes than their control-group peers, but the longer-term impact as children proceed through 
school is still an open question. To gain a better understanding of the long-term outcomes for 
the Head Start CARES children, it is important to track these children with rich data collection 
on the kinds of outcomes that are most likely to follow from these early improvements in 
emotional and social skills. 

Supporting children’s social-emotional competence is a primary focus of Head Start, 
and developmental science has provided an increasingly strong rationale for this important 
focus. This study provides evidence about the ability of various enhancements to improve 

21 Duckworth, Peterson, Matthews, and Kelly (2007). 

22 CLASS is an acronym for Classroom Assessment Scoring System. 
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children’s social and emotional skills and behaviors within the Head Start system. As such, it 
provides key information that federal policymakers and Head Start providers will need if they 
are to increase Head Start’s capacity to improve the social-emotional skills of preschool 
children. 
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